Predicting the Driver's Focus of Attention: the DR(eye)VE Project

In this work we aim to predict the driver’s focus of attention. The goal is to estimate what a person would pay attention to while driving, and which part of the scene around the vehicle is more critical for the task. To this end we propose a new computer vision model based on a multi-path deep architecture that integrates three sources of information: visual cues, motion and scene semantics. We also introduce DR(eye)VE, the largest dataset of driving scenes, enriched with eye-tracking annotations and other sensors’ measurements. This dataset features more than 500.000 registered frames, matching ego-centric views (from glasses worn by drivers) and car-centric views (from roof-mounted camera). Results highlight that several attentional patterns are shared across drivers and can be reproduced to some extent. This may benefit an ADAS system by providing indication of which elements in the scene are likely to capture the driver’s attention.